Skip to content

Fix dragon kappa bisect failing for p >> n - #388

Open
marouenbg wants to merge 2 commits into
netZoo:develfrom
marouenbg:fix-dragon-kappa-bracket
Open

Fix dragon kappa bisect failing for p >> n#388
marouenbg wants to merge 2 commits into
netZoo:develfrom
marouenbg:fix-dragon-kappa-bracket

Conversation

@marouenbg

@marouenbg marouenbg commented May 10, 2026

Copy link
Copy Markdown
Contributor

Summary

  • estimate_kappa and estimate_kappa_dragon previously called scipy.optimize.bisect with the static bracket [1.001, 1000*n]. The score-equation root scales like p*(p-1)/(4*|term_Dlogli|), which can exceed 1000*n when p ≫ n, producing the runtime error f(a) and f(b) must have different signs.
  • Add _bisect_growing_upper, a small helper that grows the upper bound geometrically until f(b) flips sign. The score equation asymptotes to a finite negative value as x → ∞, so the helper is guaranteed to bracket a root.
  • Route the five bisect call sites in dragon.py (estimate_kappa plus κ₁₁/κ₂₂/κ₁₂ and the simultaneous fit in estimate_kappa_dragon) through the new helper.

Reproducer (from the bug report)

p1 = 21337, n = 832, term_Dlogli11 = -77.92618920329457
Dlogli11(1.001)   = +2.27e11   ← positive
Dlogli11(1000*n)  = +58.87     ← still positive, bracket invalid

Asymptotic root location: p1*(p1-1)/(4*|term|) ≈ 1.46×10⁶, well above 1000*n = 8.32×10⁵. With the fix, one expansion (b → 8.32×10⁶) brackets the root and bisection converges.

Tests

  • test_catch_kappa_error (which previously asserted the failure) is repurposed to test_kappa_dragon_high_p_low_n, asserting that the same (n=10, p1=p2=1000) call now returns finite positive κ values.
  • New test_bisect_growing_upper_user_repro reproduces the exact failing parameters from the report, asserts the original call still raises, and verifies the helper recovers a root agreeing with the asymptotic prediction to <0.1%.

Test plan

  • pytest tests/test_dragon.py --deselect tests/test_dragon.py::test_dragon — 6 passed (the deselected test pulls fixtures from S3)
  • CI runs the full suite, including test_dragon

🤖 Generated with Claude Code

marouenbg and others added 2 commits May 10, 2026 12:53
The score-equation root for the kappa MLE in estimate_kappa and
estimate_kappa_dragon scales like p*(p-1)/(4*|term_Dlogli|), which can
exceed the static upper bound 1000*n when p is much larger than n.
That makes scipy.optimize.bisect fail with "f(a) and f(b) must have
different signs" because both endpoints stay positive.

Add _bisect_growing_upper, which expands the upper bound geometrically
until f(b) flips sign (the score equation asymptotes to a finite negative
value, so a root always exists), and route the five bisect call sites
through it. Repurpose test_catch_kappa_error -> test_kappa_dragon_high_p_low_n
to check the call now succeeds, and add test_bisect_growing_upper_user_repro
exercising the helper with the exact parameters from the bug report.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@katehoffshutta katehoffshutta added the ai-contrib Some or all of this code is written by AI and requires a human review. label Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-contrib Some or all of this code is written by AI and requires a human review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants